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ABSTRACT 



This thesis examines spectrometric oil analysis data from the A/E35U-3 and the 
Portable Wear Metal Analyzer (PWMA). Data from the two instruments is compared 
using simple linear regression analysis to obtain equations that can be used to convert 
the A E35U-3 values to PWMA values. The current JO A? Laboratory Manual is 
based on A/E35U-3 measurement values and is used as an aid in determining the 
condition of lubricated engine components. The lack, of a counterpart manual for the 
newly introduced PWMA makes it mandatory that values based on the A/E35U-3 have 
a conversion to values based on the PWMA. Through transformation equations to be 
developed in this paper, it will be possible for the PWMA user to utilize the current 
laboratory manual until a manual based on PWMA values is released. 
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I. INTRODUCTION 



Preventing failures in high performance aircraft engines and reciprocating engines 
due to foreseeable problems is a common sense part of the armed services mandatory 
safety programs. These programs call for regular maintenance to be done to decrease 
the opportunities for equipment failure. Through various programs it was found that 
some failure types can be anticipated by using spectrometers to trace wearmetal 
contamination levels over time in used oil samples. The actions taken based on the 
analysis of data resulted in savings from reduced failures and decreased maintenance on 
items that were not in need of scheduled periodic maintenance. 

The main spectrometric device currently in use by the armed services is the 
A/E35U-3. It is a bulky, heavy instrument that is not easily transportable to 
deployment locations. A newer device, the Portable Wear Metal Analyzer, has been 
designed to fill the need of a 'deployable' oil analyzer. Its production model 
distribution is scheduled to commence in the fall of 1987. The nature of the two 
instruments causes them to produce differing measurements of the contaminant 
wearmetals in oil samples. The differences are primarily due to their differing physics 
and engineering design. 

The measurement differences are important to those units deployed with a 
Portable Wear Metal Analyzer. Users of either instrument rely on the Joint Oil 
Analysis Program Laboratory Manual [Ref. 1J which is based on A/E35U-3 
measurements. The oil analyst uses the manual for three primary purposes: to 
compare wearmetal values obtained with the wearmetal evaluation criteria and trend 
tables; to obtain information from the table providing decision making guidance; and 
to obtain supplemental information providing additional wearmetal diagnostic guidance 
for each type of equipment evaluated. Therefore to use the manual properly, the 
analyst must have the same type values as the technical manual. Through 
transformation equations to be produced in this paper, the Portable Wear Metal 
Analyzer user will be able to convert technical manual values to those that are 
consistent with it. As usage of the new instrument increases, the benefits of including 
separate wearmetal evaluation criteria and trend tables based on the Portable Wear 
Metal Analyzer measurements will become increasingly apparent. 
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An analytical development of transformation equations between the Portable 
Wear Metal Analyzer and the A/E35U-3 will be presented in this paper. It will be 
supported by a background of the oil analysis program, descriptions of the two oil 
analysis instruments, and a discussion of the data available. The methodology of the 
analysis and a description of the models used to arrive at the final conclusions of what 
transformation equations are required to translate technical manuals from A/E35U-3 
values to Portable Wear Metal Analyzer values will be given. 
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II. OIL ANALYSIS PROGRAM 



A. BACKGROUND 

For several years, the Spectrometric Oil Analysis Program (SOAP) has been in 
use by all three military services. The Navy started a trial program in 1956 to 
determine if spectrometric analysis of oil samples could be used to predict engine 
failures. The success of the trial program led to the Navy Oil Analysis Program 
(NOAP). The inter-service importance of oil analysis was seen and culminated in the 
1976 merger of the Army, Navy, and Air Force programs into the Joint Oil Analysis 
Program (JOAP). 

Of many statements made on the purposes of the Joint Oil Analysis Program, 
Thomas Menard provides one that is simple and concise. He states that the purpose is 
to provide a continual monitoring of the amounts of wearmetal contaminants found in 
engine oil as an indicator of engine wear and possible engine failure (Ref. 2: p. 1J. The 
program technical manual defines the Joint Oil Analysis Program as: The Army, Navy, 
Air Force, Marine Corps coordinated effort to develop a standardized, mutually beneficial 
program to determine equipment wear condition through the use of oil analysis [Ref. 1: p. 

1-3). 

The armed services were long under a preventive maintenance program which 
involved set maintenance at specific intervals. Although these programs were 
successful, they were costly both in terms of excessive time and costs of periodically 
replaced maintenance items. Within the world of lubricated engines, this meant an 
aircraft or vehicle might unnecessarily be taken out of service for maintenance. In 
response to this, on-condition or reliability centered maintenance was developed. At 
the heart of this maintenance idea is the performance of maintenance tasks as required, 
vice the arbitrarily set interval schedule. The end result is increased time between 
maintenance actions. Senholzi [Ref. 3: p. 18] addressed the benefits of increased 
intervals to include reduced maintenance costs, reduced equipment downtime, and 
reduced safety hazards that may occur between maintenance intervals. The nature of 
the oil analysis program ideally suits itself to this modem approach to maintenance. 

The oil analysis program uses used oil samples for testing and thereby is able to 
non-destructively monitor the apparent contaminant content in a lubricating system. 
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This is done by measuring the wearmetal concentrations of certain key elements within 
an oil sample. The current values of this wearmetal, and the recent trend of those 
values are important in ascertaining the possible condition of a system. Through the 
use of the applicable technical manual, high levels and problem trends may be 
discovered. Determination of what action is to be taken and areas to check or 
continue to monitor are recommended by the technical manual. Unfortunately, the oil 
analysis program is not a cure-all and will not discover every type of failure or 
guarantee to find a problem trend every time. 

The oil analysis program has vastly increased over the years as economics and 
costly, sensitive components have played larger roles. Currently, the primary oil 
analysis instrument is the A/E35U-3 (hereafter called the Dash-3). It is in use 
worldwide by the armed services and other groups, and plays a large role in the 
maintenance previously described. This is demonstrated by the 1.5 million samples 
analyzed per year by the 100 plus Air Force JOAP laboratories alone [Ref. 4]. Due to 
its weight, size, and supporting peripherals required, the Dash-3 is normally used in 
stationary laboratories (including aircraft carriers). 

In many areas, and for many engine types, the need for analysis is immediate in 
order to keep planes in the air and vehicles on the road. Restrictions on program 
usefulness are therefore evident if oil samples were required to be shipped to distant 
laboratories for analysis and have the corresponding delay in return of the subsequent 
results. The resulting requirements for a transportable oil analysis machine led to the 
Portable Wear Metal Analyzer (hereafter called the PWMA). It is a lightweight 
instrument that is easily transported utilizing two small cases. Its design makes it ideal 
for units that are deployed away from a nearby JOAP laboratory and which still need 
quick, reliable results. In the following sections, the principles of operation of both 
instruments will be discussed. 

B. DISCUSSION OF INSTRUMENTS USED IN ANALYSIS 

The measurements of two different types of spectrometers are compared in this 
analysis. The difference in their measurements is rooted in the different physical 
principals they employ, atomic emission and atomic absorption spectrometry. The 
following is provided as background for the physical concepts and uses of these two 
instruments. 
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1. A/E35U-3 

The A/E35U-3 (Dash-3) fluid analysis spectrometer is the standard instrument 
tor the Joint Oil Analysis Program today. It is an atomic emission spectrometer and is 
manufactured by the Baird Corporation of Bedford, Massachusetts. It is a self- 
contained unit that weighs 800 pounds and has dimensions of 60 x 52.5 x 49.5 inches. 
Within its shipping container, it weighs 1400 pounds and has dimensions of 74 x 67 x 
47. 7o inches [Ref. 1: p.4-2]. The instrument has the capability of measuring twenty 
elements within an oil sample. These twenty elements are listed in Table 1 . The 
measurement of all elements is accomplished simultaneously. 



TABLE 1 

ELEMENTS MEASURED BY THE A/E35U-3 



Fe Iron 
Ag Silver 
AT Aluminum 
Be Bervllium 
Cr Chromium 
Cu Copper 
Mg Masnesium 



Na Sodium 
Ni Nickel 
Pb Lead 
Si Silicon 
Sn Tin 
Ti Titanium 
B Boron 



Ba Barium 
Cd Cadmium 
Mn Manganese 
Mo Molvbdenum 
V Vanadium 
Zn Zinc 



Atomic emission spectrometers are optical type instruments used to determine 
the concentration of wearmetals in lubricating fluid. The analysis is accomplished by 
subjecting ("burning") the sample to a high voltage spark which energizes the atomic 
structure of the metallic elements, causing the emission of light. Within the Dash-3, 
this is done through the use of a rotating disk electrode spark source. The light is 
caused by the atomic structures of the elements being excited into higher energy states 
by the high temperature and then returning to their ground state through the emission 
of light. One or more characteristic wavelengths may be given off by each element. 
The emitted light is subsequently focused into the optical path of the spectrometer and 
separated according to wavelength, converted to electrical energy and then measured. 
The emitted light for any element is proportional to the concentration of wearmetal 
suspended in the lubricating fluid. This makes possible the estimates for the wearmetal 
concentration levels for each element that are normally measured in parts per million 
(ppm). [Ref. 1,5: pp. 2-1, 21] 

The final values obtained from a spectrometer are subject to many sources of 
error. Several of these sources apply to both the Dash-3 and the PWMA. Some of the 
reasons for error and factors that may influence the actual value obtained are given 
below [Ref. 1: pp. 2-3, 2-4]: 
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• Oil sample from wrong component 

• Contaminating substances in system 

• Oil is not homogeneous in content 

• Type of instrument being used, emission or absorption spectrometer 

• Contaminated! poor quality calibration standards 

• Changes of rod or disc electrodes without restandardizing equipment 

• Metallic components in fuid additives within oil manufacture 

• Electrolytic corrosion due to seasons, storage 

• Dirt and sand contamination 

• Metal content of the fuel 

• Break-in period of new or recently overhauled components 

Another possible problem source is that of using synthetic ester oil to 
calibrate the instrument. Rhine [Ref. 6: p. 39] found this to directly effect the Dash-3 
by the enhancement of emission from metals in an ester oil matrix relative to metals in a 
hydrocarbon matrix. The end product of this is superficially high readings from the 
ester oil when calibration is done with a hydrocarbon oil. Similarly, problems are 
introduced by calibrating an instrument using standards prepared in a synthetic oil 
matrix. Specific to the Dash-3 design, errors may be introduced if there are 
inefficiencies in the particle transport capability of the rotating disk electrode, or in the 
vaporization and excitation of those transported particles at the arc/spark source 
[Ref. 6: p. 53]. A study for the Air Force by United Technologies Corporation placed 
some of the precision problems of the Dash-3 on the variable excitation characteristics 
of direct current (dc) arcs operated in air, inaccuracies due to variations in sample 
viscosity, and the inability to analyze particles larger than 10 micrometers in size 
[Ref. 7: p.l], 

2. PWMA 

The Portable Wear Metal Analyzer (PWMA) is planned as the standard 
instrument for deployments to remote areas where the Dash-3 is not located or 
suitable. It is a graphite furnace atomic absorption spectrometer manufactured by the 
Perkin-Elmer Corporation's Applied Science Division of Pomona, California. It is 
composed of two self-contained units that double as shipping containers. The units are 
lightweight, weighing 60 pounds and 40 pounds respectively, and have identical 
dimensions of 11 x 18 x 18 inches [Ref. 8: p. 1-4]. The instrument has the capability of 
measuring nine elements within an oil sample, these elements are listed in Table 2 . 
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The measurements of all elements is accomplished simultaneously. The instrument has 
repeatability specifications of + /- lppm or +/- 2.5% of full scale, whichever is greater. 



Atomic absorption spectrometers are similar to their emission counterparts in 
being optical type instruments used to determine the concentration of wearmetals in a 
lubricating fluid. However, the physical concepts of the machines are very different. 
With the atomic absorption spectrometer, the atomic structures of the elements present 
become energized enough by high temperatures to absorb light (rather than emit light 
as in the Dash-3). Within the PWMA, the heating is done in a graphite furnace tube 
whereas other atomic absorption spectrometers usually use a flame. The graphite 
furnace atomic absorption spectrometer uses the graphite tube as a resistive heating 
element to atomize the wearmetals. Light energy having the same ("characteristic") 
wavelength of the element being analyzed is then radiated through the graphite tube. 
Two multi-element hollow-cathode lamps are used for this in the PWMA. The 
resultant light is converted to electrical energy and measured electronically by using 
photomultiplier detectors. The amount of light energy absorbed by the elements in the 
optical path of the graphite tube is proportional to the concentration of wearmetal 
suspended in the lubricating fluid. This makes possible the estimates for the wearmetal 
concentration levels for each element that are normally measured in parts per million. 
The after to before ratio of light intensities that is used makes the signal less 
susceptible to intensity variations of the spectra caused by the environment. 
[Ref. 1,4: pp 2-2,6] 

Six prototype PWMA instruments were acquired by the Air Force for testing 
in an operational environment with military personnel as operators [Ref. 9: p. 2]. Field 
test plans were developed by the Air Force Wright Aeronautical Laboratories 
(AFWAL). As part of this plan, instruments were provided to four locations to be 
used alongside the Dash-3. Locations in receipt of these instruments were: Naval Air 
Rework Facility (NARF), Pensacola; Elmendorf AFB; Langley AFB; and Myrtle Beach 
AFB. Preliminary testing on a prototype PWMA showed it to be successful in 



TABLE 2 

ELEMENTS MEASURED BY THE PWMA 



Fe Iron 
A 2 Silver 
Af Aluminum 



Cr Chromium 
Cu Copper 
Mg Magnesium 



N'i Nickel 
Si Silicon 
Ti Titanium 
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measuring particles up to 20 micrometers in size [Ref. 4: p. 60). The prototype 
PWMA's currently use preprogrammed algorithms in calibrating themselves. This is 
done using three successive calibration runs. The runs specify three points for each 
element's calibration curve algorithm and may result in small inaccuracies. Further 
areas of possible error are described in Reference 4 . 

Initial testing of the PWMA has shown it to yield values that are 
approximately one-half of those obtained with the Dash- 3. For example, a sample that 
measures 11 ppm on the Dash-3 may only measure 6 ppm on the PWMA. The 
wearmetal concentration level (in ppm) is readily seen from this to depend on which 
instrument is used. Investigation of this point shows the controlling factor to be in 
whether the measurement is based on atomic emission (as with the Dash-3), or based 
on atomic absorption (as with the PWMA). This difference in using light emission or 
light absorption for estimating an element concentration level leads to the separate 
scales obtained. 
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III. data 



A. FIELD USE OF JOAP DATA 

Once values have been obtained From an oil analysis spectrometer, an analyst 
must be able to do more than add them to a database. His primary goal is to 
determine if there is any abnormal condition present or developing in the parent 
component of the sample. This goal may be achieved by following the wearmetal 
methodology presented by the JOAP Laboratory Manual. The major parts of the 
method are [Ref. 1: p.6-1]: 

• a wearmetal evaluation and trend table 

• a table providing decision making guidance 

• supplemental information providing additional wearmetal diagnostic guidance for 
each type of equipment evaluated 

When consumption and replenishment of fluid is continuously balanced in a 
component, the wearmetal concentration within the lubricant tends to increase after a 
fluid change until a steady state level is achieved. This effect is shown in Figure 3.1 . 
The wearmetal concentration steady state point in this model is thus a function of two 
variables: rate of fluid consumption and replenishment; and the rate of wearmetal 
production by internal friction within the equipment. When the fluid is added or 
changed periodically, as is done with many items of equipment, a sawtooth pattern 
emerges. Such a pattern is shown in Figure 3.2 . [Ref. 1: pp. 2-5,6) 

Because of a tendency toward a steady state level, it is not only important to 
know the actual wearmetal concentration, but also to know the rate of increase (trend) 
of that concentration. Increased sampling is always recommended as levels increase in 
order to decrease the probability of a wearmetal concentration going from normal to 
abnormal without an intermediate sample. The exact procedure the oil analysts are 
trained to follow in evaluating a sample is a logical sequence of actions. The JOAP 
manual mandates the following procedure [Ref. 1: p. 6-2): 

• Analyze the sample and obtain the wearmetal concentration results. 

• Determine the range where each critical wearmetal falls from the appropriate 
wearmetal evaluation criteria and trend table. 

• Compare the wearmetal concentration ranges of the current sample with the 
ranges of the last previous sample. 

• Determine the wearmetal trend ( trend value for a ten hour period). 
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• If a maintenance action is indicated, consult the supplemental information 

provided. 




HOURS 



Figure 3.1 Theoretical Concentration with Constant Fluid Replenishment. 




HOURS 



Figure 3.2 Concentration with Periodic Fluid Addition or Change'. 

Fable 3 [Ref. 1: pp. 9S,99| is provided to gain a fuller understanding of the oil 
analysts' procedure. It provides the wearmetal evaluation criteria and trend table for 
the T64 scries engine commonly used in the C11-53A, D helicopter. A sample scenario 
would show the analyst checking for a normal trend by comparing the current values 
with those of the previous sample, and then checking for to see if the current values are 
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in a normal range. With that information he would-enter a decision making guidance 
table and obtain from it the advice codes it lists. This decision aid is shown in Table 4 . 
When the code indicates anything abnormal, some type of action is recommended. 
Normally the initial action is to request a 'red-tagged' sample and to cease operation of 
the equipment. The 'red-tagged' sample is to gain confirmation of the abnormal results 
before any type of maintenance action will be recommended. If abnormal results are 
confirmed, the supplemental information provided in the technical manual is consulted 
and a maintenance action is recommended. The supplemental information may be very 
useful in pin-pointing possible problem areas and identifying a specific recommendation 
to the sample originator [Ref. 1: p. 6-1]. 

An example of the above procedure using the tables shown is given here. An oil 
analyst receives a sample from an engine that was previously sampled twenty engine 
hours earlier. His logbook shows that its last sample was normal and had values as 
follows (in ppm): Fe-10, Ag-0, Al-2, Cr-0, Cu-1, and Mg-1. The present sample is 
tested with the following results: Fe-19, Ag-4, Al-6, Cr-0, Cu-1, and Mg-1. The analyst 
determines the ten-hour trend values to be (in ppm/hour): Fe-4.5, Ag-2, Al-2, and the 
remainder zero. Using Table 3, he would discover Fe and Ag to have abnormal trends 
and high ranges, with all other elements normal. Entering Table 4 in the high range in 
column one and reading across through the normal range in column two for the 
previous sample, and then through the abnormal trend in column three, one obtains 
the advice code P from column four. The analyst translates this code as: do not 
fly; operate', do not change oil, submit sample ASAP and relays this to the sample 
originator. The 'red-tagged' sample is subsequently analyzed for verification of the last 
results. If results are identical to the previous ones, the appropriate advice code from 
column five is utilized. This might include using the lower part of Table 3 to determine 
possible maintenance problems such as the power turbine shafts. 

B. DISCUSSION OF DATA USED 

The data in this analysis was obtained from a field test for the evaluation of the 
Portable Wear Metal Analyzer that was conducted in 1985. The data was collected 
over a two month period from four locations: NARF, Pensacola, Elmendorf AFB, 
Langley AFB, and Myrtle Beach AFB. Both Dash-3's and prototype PWMA's were in 
the laboratories at each site in order to provide data for their comparisons. Only the 
nine elements that the PWMA is able to measure were recorded for the Dash-3. 
Though the primary’ objective of the field test was to determine the functionality of the 
PWMA [Ref. 10: p.l], the data obtained easily lent itself to this study. 
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TABLE 3 

SAMPLE WEARMGIAL EVALUATION CRITERIA AND TREND TABLE 



ENGINE: T64-6/.7/-7AM0/-12M13M15 

AIRCRAFT: (CH-53A/D) (RH-53A/D) (VH-53D) (UC-8A) 



A/F35C-3 





Fe 


Ag 


Al 


Cr 


Cu 


Mg 






Abnormal Trend 
(PPM Increase 
in 10 hrs) 


4 


2 


4 


2 


3 


2 






Normal Range 


0-14 


0*2 


0-10 


0-2 


0-6 


0-5 






Marginal Range 


1517 


N/A 


11-12 


3 


7 


6 






High Range 


18-21 


3 


13-14 


4 


8-9 


7 






Abnormal 


22* 


4* 


15> 


5* 


10* 


8* 







Average Concentration Other Elements: 

Nl-1 Pb-2 Si-3 Sn-7 Tl-1 Mo-1 



ENGINE: T64-6/-7/-7A/-12M 13M 15 (Cont) 

AIRCRAFT: (CH-53A/D) (RH-53A/D) (VH-53D) 



[ID 




Main bearing balls, rollers and races, accessory gearbox gears and shafts, FTO gears 


dD 


Ag 


Accessory gearbox bearings 


|Fe 


A iJ 


Power turbine shafts 


1 F * 


— Cul 


FTO bearings 


I 

0 


J2iJ 


Main bearmg cagas 

Lube and scavenge oil pumps and thermal gradient housing 



NOTE 

Thermal gradient housing could be pnme source of | Al| especially after heavy use of 
engine anti-idnf system. 
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DECISION MAKING GUIDANCE 



TABLE 4 

DECISION MAKING GUIDANCE TABU 
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0 


03 

O 


00 

ha 
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00 


00 

h* 

0 


00 

ha 

O 


00 


Cu 


r J 


Cu 


U 


Cu 


03 

O 


Cu 
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0 


CU 


UJ 


cu 
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0 

c 

X 
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0 
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13 

g 

0 
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Normal 


Marginal 


High 


Abnormal 




a 

§ 
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Marginal 


X 

Of 

5 


Abnormal 




a 

0 


Marginal 


High 


Abnormal 




a 
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Marginal 


High 
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RANGE, 
THIS SAMPLE 


NORMAL 


MARGINAL 


HIGH 


ABNORMAL 
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As designed, the test called for each site to use six graphite tubes for 160 sample 
'bums' each. Many samples were to be tested five consecutive times when analyzed. 
The test plan was designed for repetitive four day cycles. The sequence of events it 
followed for the PWMA is shown in Table 5 (Ref. 10: p. 11]. The sequence of events 
for the Dash-3 was nearly identical to that of the PWMA. Consistency in following 
this test plan varied among the different locations. This directly affected the number of 
sample results provided by each location. A compilation of the number of samples 
used in this analysis from each location is given in Table 6 . 

TABLE 5 

PWMA FIELD TEST BURN SEQUENCE 



Day 1 


Calibration Runs 
Verification Samples 
Correlation Samples 1, 2, 3 
Random Samples 
Verification Samples 


Day 2 


Reslope 

Verification Samples 
Correlation Samples 4,5 
Random Samples 
Calibration Check Samples 
Random Samples 
Verification Samples 


Day 3 


Calibration Samples 
Verification Samples 
Correlation Samples 6,7 
Random Samples 
Verification Samples 


Day 4 


Reslope 

Verification Samples 
Correlation Samples 8,9 
Random Samples 
Calibration Check Samples 



As noted in Table 5, there were four main classes of samples tested with the 
instruments: calibration, verification, correlation, and random samples. Calibration 
samples of MIL-L-7808 oil were used for the internal calibration of the PWMA. Three 
calibration samples containing 20%, 50%, and 100% respectively of full scale values 
for all nine metals were used for the calibration runs and were also used for quality 
assurance in the calibration check runs. Verification samples were made in MIL- 
L-7808 oil and contain concentration levels of 10%, 40%, and 70% of the top of the 
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TABLE 6 

NUMBER OF SAMPLES USED IN ANALYSIS 





Elm 


Lan 


Myr 


Pen 


Correlation Samples 


53 


26 


24 


80 


Verification Samples 


65 


33 


27 


39 


| ... , 1 

Total Samples 


118 


59 


51 


119 



dynamic range. Correlation samples were designed to reflect the pseudo-realism of 
having various concentration levels for each element. Nine different correlation 
samples were used in the test. Because of the inavailability of MIL-L-7808 oil with 
significant wearmetal levels for the correlation samples, used MIL-L-23699 turbine 
engine oil was utilized after blending with organo-metallic concentrates as necessary by 
JOAP-TSC. These samples were ensured to contain a minimum of seven wearmetals. 
The random samples were routine daily samples obtained at each site and were 
typically MIL-L-7808 lubricants from the Air Force, and MIL-L-23699 lubricants from 
the Navy. Reslopes were calibration checks using a 50% of full scale calibration 
sample. (Ref. 10: p. 6] 

Due to the nature of the calibration runs and the low values in the random 
samples, only verification and correlation samples were used in this analysis. This 
made possible a total of twelve samples, consisting of three different verification and 
nine different correlation samples. Sample numbers (or levels as appropriate) were 
kept in the database as well as the graphite tube number in which a sample was used 
for later use in determining their means. Each sample 'bum' with the PWMA 
correlation samples was repeated five consecutive times. In order to provide a one to 
one correspondence of data between the instruments, the average of the consecutive 
bums' was used in the analysis. The PWMA value (average value for correlation 
samples or actual measurement for verification samples as appropriate) was coupled 
with its counterpart on the Dash-3 that was closest in time. In normal cases this led to 
results such as pairing the first verification sample 'level 10' measured during the day 
from each instrument. Differences in sample sizes for the various element-location 
combinations was a result of missing data points and the removal of extreme outliers 
from the data. 
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IV. ANALYSIS 



A. OBJECTIVES 

The objective of this analysis is to find a functional relationship between Dash-3 
and PWMA measurements for the nine elements that the PWMA measures. Analysis 
of variance techniques will be used in order to determine what effects cause significant 
differences in the measurements obtained. Regression analysis will be used to yield the 
parameters of an equation to relate the instrument measurements, and to obtain the 
standard deviations of those parameters. The coefficient of determination or r 2 statistic 
will be used as a measure of how well the model fits the data and to compare the 
different models. It is also known as the square of the coefficient of determination and 
represents the percentage of total variation explained by the model. 

B. SCATTERPLOTS 

Scatterplots of the data are useful in determining areas for further investigation. 
They were used in this analysis to provide insight into the rejection of the hypothesis 
that there is no difference in measurements due to location. The rejections are 
described in the analysis of variance section. The scatterplots in Figures 4.1 - 4.9 are 
location coded and include all verification and correlation sample data points. As 
reflected in Table 7, examination of these plots show Pensacola with higher values and 
Elmendorf with lower values for most elements (seven of nine in each case). The 
consistent extremes shown by these two locations are a primary reason for differences 
found between locations. The number of samples submitted does not explain the 
extremes as both had larger sample sizes. Coded scatterplots of the graphite tube 
furnaces for each location/element combination are shown in Appendix A. The tubes 
were reviewed in particular because they were prototypes and not production models. 
The quality controls that will be used in manufacturing the production models was not 
used for the prototypes, consequently, it was suspected that they could be a large 
source of variability in the measurements. The graphite furnace tubes were provided to 
each site with the tubes labeled sequentially within sites. Some tubes malfunctioned 
and could not be used due to calibration failures, causing skips in the tube numbers 
reported. Most tubes did not last as long as expected originally so all tubes did not 
have an equal number of burns. With minor exceptions, no unusual characteristics 
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PWMA PWMA 



FE IN PPM 




DASH -3 

figure 4.1 Location Coded Scattcrplot of Fe Using All Points. 



AG IN PPM 




Figure 4.2 Location Coded Scatterplot of Ag Using All Points. 
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PWMA 



AL IN PPM 




Figure 4.3 Location Coded Scattcrplot of A1 Using All Points. 



CR IN PPM 




Figure 4.4 Location Coded Scattcrplot ol'Cr Using All Points. 
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PWMA 



CU IN PPM 




Figure 4.5 Location Coded Scatterplot of Cu Using All Points. 



MG IN PPM 




Figure 4.6 Location Coded Scatterplot of Mg Using All Points. 
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PWMA PWMA 



Nl IN PPM 




Figure 4.7 Location Coded Scattcrplot of Ni Using All Points. 



Si IN PPM 



o 



o 

CN 



o 



o 




DASH- J 



C-ELMfNDOnr 

L-I>NCIE> 

U .fcfiRH r OLACH 
• r.-.PE NMCOO 



Figure 4.8 Location Coded Scattcrplot of Si Using All Points. 
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Tl IN PPM 




Ligurc 4.9 Location Coded Scatterplot of Ti Using All Points. 



TAB UP: 7 

SUMMARY OF LOCATION CODliD SCATTLRPLO I S 



Element Location Coded Results 



l'e 


P high 


H low 


Ag 


P high 


L low 


A1 






Cr 




li low 


Cu 


P high 


K low 


Mg 


P high 


I: low 


Ni 


P high 


L low 


Si 


P high 




Ti 


P high 


1: low 



were found. A summary of the graphite furnace tubes in Table 8 shows Llmendorf 
used six different graphite tubes in its experiments with tube I measuring higher than 
normal in seven elements, tube 7 high in five elements, atid three points of tube 5 high 
in seven elements (indicated by 5,). Though the three points from tube 5 appeared to 
be an aberration, they were included in the analysis. Higher, lower than normal is not 
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TABLE 8 

SUMMARY OF GRAPHITE TUBE CODED SCATTERPLOTS 





Elm 


Lan 


Myr 


Pen 


Fe 


1,5 3 ,7 Hi 


5 Hi 7 Lo 




1 Hi 


Ag 


1,5 3 Hi 




3 Lo 


16 Hi 


A1 


1 Hi 








Cr 


1,5 3 Hi 


1 Hi 




1 Hi 


Cu 


1,5 3 Hi 


3 Hi 7 Lo 






Mg 


7 Hi 


7 Lo 






Ni 


1,5 3 ,7 Hi 








Si 


5 3 ,7 Hi 








Ti 


1,5 3 ,7 Hi 


2 Lo 




1 Lo 


Tubes 

Used 


1,2, 3, 4 
5*7 


1,2,3, 5 


1,2, 3, 4 

5,6 


l .3.4.5 
6,&Al2 
li,i5 16 



used in a statistical sense but as a subjective term. A symbol that characteristically 
plotted as a maximum/minimum value across the range of x values was given the 
appropriate subjective term. The five tubes used by Langley showed tube 7 measuring 
low in three elements, tube 2 low in one, and tubes 1,3, and 5 high in one element 
apiece. Myrtle Beach used six tubes and showed nothing unusual except for tube 3 
measuring low in one element. Pensacola provided the most data and used the most 
tubes, 11. Its scatterplots are not unusual except for tube 1 being high for two 
elements and low for one,, and tube 16 being high for one element. In summary, the 
scatterplots indicate that the locations provide different measurements, with Pensacola 
normally high, and Elmendorf normally low. The graphite furnace tubes do appear to 
have a statistically significant difference within each instrument, but not a practical 
difference. 
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C. ANALYSIS OF VARIANCE 

As stated, a superior tool to scatterplots iri determining if various effects are 
statistically significant is analysis of variance. It is used here to determine if the 
various instruments (the four prototype PWMA's and the four Dash-3's) are providing 
measurements close enough to be considered not statistically different. Examples of 
the various AXOVA models applicable can be found in References 11 and 12 . Using 
the standard assumptions of normality, an alpha level of significance of 0.05 is used 
throughout this paper. Statistical significance obtained may be more restrictive in its 
determinations than what would be considered a practical significance by the oil 
analysts. In other words, the numbers may be close enough to be considered the same 
by the analysts. No assertions as to what this practical significance might be will be 
addressed. Analyses of variance were run on the correlation samples to determine if 
location, sample, or their interaction is significant. The resulting tables and associated 
p-values are given in Appendix B. As summarized in Table 9, location differences 
were found with all elements except Xi on the Dash-3. Location-sample interactions 
were significant with four elements for the Dash-3, and in all but two elements for the 
PWMA. These results imply greater variability in the prototype PWMA instruments. 



TABLE 9 

SUMMARY OF CORRELATION DATA AXOVA 
PWMA Dash-3 





LOC 


SAM 


I XT 




LOC 


SAM 


I XT 


Fe 


X 


X 


X 




X 


X 


X 


Ag 


X 


X 


X 




X 


X 




A1 


X 


X 


X 




X 


X 




Cr 


X 


X 


X 




X 


X 




Cu 


X 


X 






X 


X 


X 


Mg 


X 


X 


X 




X 


X 




Xi 


X 


X 








X 


X 


Si 


X 


X 


X 




X 


X 




Ti 


X 


X 


X 




X 


X 


X 



X denotes statistically significant at a = .05 



The analyses of variance on the verification sample data for both instruments is 
provided with their corresponding p-values in Appendix C, and a summary given in 
Table 10 . Table 10 shows that when interactions were considered, location was not 
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significant for three PWMA elements (Cu, Si, and Ti) and one Dash-3 element (Ni). 
Analysis of interactions showed all PWMA elements except Mg did not have 
significant interactions, while all Dash-3 elements but Ni did have a significant 
interaction. In contrast to the correlation data, this shows a strong variability in the 



TABLE 10 

SUMMARY OF VERIFICATION DATA ANOVA 
PWMA Dash-3 





LOC 


LEV 


INT 




LOC 


LEV 


INT 


Fe 


X 


X 






X 


X 


X 


Ag 


X 


X 






X 


X 


X 


A1 


X 


X 






X 


X 


X 


Cr 


X 


X 






X 


X 


X 


Cu 




X 






X 


X 


X 


Mg 


X 


X 


X 




X 


X 


X 


Ni 


X 


X 








X 




Si 




X 






X 


X 


X 


Ti 




X 






X 


X 


X 



X denotes statistically significant at a = .05 



Dash-3 compared to the prototype PWMA. As expected and designed, the different 
verification levels and different correlation samples were significantly different within 
their respective groups. Further analysis on the verification data is summarized in 
Table 11 , which takes a closer view of location effects at each of the three levels. 
Recall that these levels represent the samples which contain 10, 40, and 70 percent of 
the full scale dynamic range for each element. The table gives each instrument's 
corresponding element-level matrix of significant location effects. The matrix shows 
the prototype PWMA to be very consistent in its measurements across locations as 
opposed to the Dash-3, which lacks consistency for most elements and levels. The 
basis for this statement lies in the few holes shown in the Dash-3 matrix and the few 
marks in the PWMA matrix. Examination of location effects on the verification data 
was also done using means. The highest and lowest location mean was recorded for 
each element and is shown in Table 12 . From this it can be seen that a location that 
is high(low) for an element is usually high(low) for all three levels for that element. Of 
interest is that the location that measures high(low) for an element on one instrument 
will not normally be high(low) on the other instrument as well. This shows an 
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TABLE 11 

SUMMARY OF VERIFICATION LEVEL ANOVAS 



PWMA Dash-3 





10 


40 


70 




10 


40 


70 


Fe 






X 




X 


X 


X 


A 2 










X 


X 


X 


At" 




X 






X 


X 


X 


Cr 


X 




X 




X 


X 


X 


Cu 














X 


Mr 




X 


X 






X 


X 


Ni 
















Si 










X 


X 


X 


Ti 










X 


X 





X denotes statistically significant at a= .05 



apparent low correlation between the two instruments regarding the same location 
being high(low) for a particular element. 



TABLE 12 

VERIFICATION LEVEL HIGHEST AND LOWEST MEANS 



PWMA Dash-3 





10 


40 


70 




10 


40 


70 


Hi 


P 


P 


L 




M 


L 


E 


Fe Lo 


M 


E 


E 




P 


P 


P 


Hi 




M 


M 




L 


L 


L 


Ag Lo 




L 


P 




P 


P 


P 


Hi 


P 


M 


L 




L 


E 


E 


A1 Lo 


E 


E 


E 




P 


P 


P 


Hi 




M 


L 




P 


L 


L 


Cr Lo 




E 


E 




M 


M 


M 


Hi 


P 


P 


P 






P 


L 


Cu Lo 


E 


E 


E 






M 


M 


Hi 


L 


L 


L 




P 


P 


L 


Mg Lo 


P 


P 


P 




E 


E 


E 


Hi 




P 


L 








M 


Ni Lo 




M 


M 








L 


Hi 


E 


E 


L 






E 


L 


Si Lo 


P 


P 


P 






M 


M 


Hi 


M 


L 


L 






L 


L 


Ti Lo 


P 


P 


P 






P 


P 
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D. REGRESSION 

Regression is used as the principal tool to determine functional relationships 
(transformation equations) between the measurements of the two different instruments. 
Tables 13-21 show for each element the results of several regressions run on the data. 
For each regression, the following are given: r 2 , the intercept and its standard 
deviation, the slope and its standard deviation, and the number of data points used in 
the regression. Two principal models were analyzed, the first model used all data 
points from all locations regardless of the location sample sizes (model 1); and the 
second model used the means of the samples and levels at each location, thereby 
allowing equal sample sizes (model 2). The various sample sizes w r ere presented in 
Table 6 . Appendix D gives a basic regression model. The majority of regressions 
done used the simple linear model of: 

PWMA = B 0 + B l (DASH- 3) + e . (eqn4.1) 

Other regressions used a quadratic fit or used simple linear fits on transformations of 
the data. No attempts were made to force the resulting equations through the origin. 

An ideal regression using the simple linear model with least squares estimators 
has a high r 2 . It will also meet the assumption of constant variance across the range of 
interest (homoscedascity), and that of normalized residuals. The usual least squares 
regression estimators have optimal properties if one assumes that the variance of the y 
values remains constant as x increases, together with 

Efy] = a + bXj . (eqn 4.2) 

Many measuring instruments have variances which increase with the magnitude of the 
quantity measured. Letting <J 2 represent the variance of y., one could assume a simple 
linear relation 

<y 2 = kXj , (eqn 4.3) 

in which case Equation 4.2 can be transformed by dividing by Vxj . This will give a 
constant variance model and help to normalize the residuals. However, this procedure 
will not work if the data displays a variance that first increases and then decreases as 
the true content increases. 
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TABLE 13 

REGRESSION RESULTS FOR FE 



— 

Data 


Model 


1 

T* 


B 0 


% 


B l 


% 


n 


Elmendorf 


1-Lin 


.81 


-2.68 


1.55 


.527 


.024 


116 


Langley 


1-Lin 


.85 


-3.02 


2.17 


.618 


.035 


58 


Myrtle Beach 


1-Lin 


.96 


-4.57 


1.17 


.629 


.018 


51 


Pensacola 


1-Lin 


.95 


-1.75 


.85 


.667 


.014 


116 


All Loc 


1-Lin 


.87 


-2.43 


.79 


.596 


.013 


341 


LanMyrPen 


1-Lin 


.92 


-2.47 


.76 


.637 


.013 


231 


All Loc 


1-Quac 


. .87 


* 


* 


* 


* 


341 


















Corr Means 


2-Lin 


.87 


-.36 


1.64 


.507 


.034 


36 


Verif Means 


2-Lin 


.97 


-9.76 


3.01 


.719 


.040 


12 


All Loc Means 


2-Lin 


.90 


-1.92 


1.59 


.570 


.028 


48 


LMP Means 


2-Lin 


.96 


-2.09 


1.33 


.617 


.022 


36 


PWMA 2 -Dash-3 2 


2-Lin 


.82 


* 


* 


* 


* 


48 


PWMA-Dash-3 2 


2-Lin 


.85 


* 


* 


* 


★ 


48 


PWMA 2 -Dash-3 


2-Lin 


.77 


* 


* 


* 


* 


48 



^-denotes not included due to little or no r 2 improvement. 
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TABLE 14 

REGRESSION RESULTS FOR AG 



Data 


Model 


r 2 


B 0 


* B o 


B i 


ffB l 


n 


Elmendorf 


1-Lin 


.74 


.49 


.27 


.466 


.026 


118 


Langley 


1-Lin 


.92 


-.37 


.26 


.574 


.022 


58 


Myrtle Beach 


1-Lin 


.88 


.22 


.32 


.535 


.028 


51 


Pensacola 


1-Lin 


.92 


.24 


.20 


.685 


.019 


117 


All Loc 


l-Lin 


.82 


.29 


.15 


.559 


.014 


344 


LanMyrPen 


1-Lin 


.87 


.19 


.17 


.605 


.015 


229 


All Loc 


1-Quac 


.84 


* 


♦ 


* 


* 


344 


















Corr Means 


2- Lin 


.91 


.02 


.37 


.539 


.029 


36 


Verif Means 


2-Lin 


.95 


-.45 


.38 


.690 


.050 


12 


All Loc Means 


2-Lin 


.91 


.11 


.30 


.542 


.025 


48 


LMP Means 


2-Lin 


.91 


.16 


.36 


.567 


.030 


36 


PWMA 2 -Dash-3 2 


2-Lin 


.81 


* 


* 


* 


* 


48 


PWMA-Dash-3 2 


2-Lin 


.76 


♦ 


♦ 


* 


♦ 


48 


PWMA 2 -Dash-3 

! 


2-Lin 


.83 


* 


* 


* 


* 


48 



"-denotes not included due to little or no r 2 improvement. 
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TABLE 15 

REGRESSION RESULTS FOR AL 



Data 


Model 


r 2 


B 0 


<Tr> 

B 0 


B i 


% 


n 


Elmendorf 


1-Lin 


.60 


.88 


.73 


.428 


.032 


116 


Langley 


1-Lin 


.67 


1.65 


1.00 


.650 


.060 


58 


Myrtle Beach 


1-Lin 


.80 


2.14 


.80 


.618 


.045 


48 


Pensacola 


1-Lin 


.59 


5.25 


.70 


.399 


.031 


118 


All Loc 


1-Lin 


.57 


3.19 


.44 


.442 


.021 


340 


LanMyrPen 


1-Lin 


.31 


7.89 


.55 


.250 


.025 


231 


All Loc 


1-Quac 


.59 


* 


* 


it 


it 


340 


















Corr Means 


2-Lin 


.75 


3.21 


1.10 


.435 


.045 


34 


Verif Means 


2-Lin 


.84 


1.00 


1.30 


.723 


.098 


12 


All Loc Means 


2-Lin 


.75 


3.15 


.87 


.451 


.039 


46 


LMP Means 


2-Lin 


.79 


3.74 


.93 


.482 


.043 


34 


PWMA 2 -Dash-3 2 


2-Lin 


.65 


* 


* 




it 


46 


PWMA-Dash-3 2 


2-Lin 


.60 


* 


♦ 


* 


♦ 


46 


PWMA 2 -Dash-3 


2-Lin 


.69 


* 


* 


it 


it 


46 



•-denotes not included due to little or no r 2 improvement. 
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TABLE 16 

REGRESSION RESULTS FOR CR 



Data 


Model 


r 2 


B 0 


% 




% 


n 


Elmendorf 


1-Lin 


.70 


.58 


.23 


.540 


.033 


118 


Langley 


1-Lin 


.91 


-.19 


.26 


.788 


.033 


59 


Myrtle Beach 


1-Lin 


.92 


.48 


.23 


.804 


.036 


49 


Pensacola 


1-Lin 


.96 


-.19 


.12 


.809 


.016 


119 


All Loc 


1-Lin 


.83 


.20 


.12 


.711 


.017 


345 


LanMyrPen 


1-Lin 


.93 


.03 


.11 


.792 


.015 


229 


All Loc 


1-Quac 


.84 


* 


* 


* 


* 


345 


















Corr Means 


2- Lin 


.90 


.21 


.35 


.738 


.042 


36 


Verif Means 


2- Lin 


.95 


-.23 


.33 


.873 


.060 


12 


All Loc Means 


2- Lin 


.90 


-.06 


.28 


.738 


.036 


48 


LMP Means 


2-Lin 


.97 


-.29 


.20 


.822 


.026 


36 


PWMA 2 -Dash-3 2 


2- Lin 


.81 


* 


* 


* 


* 


48 


PWMA-Dash-3 2 


2-Lin 


.83 


* 


* 


♦ 


♦ 


48 


PWMA 2 -Dash-3 


2-Lin 


.80 


* 


* 


* 


* 


48 



^-denotes not included due to little or no r 2 improvement. 
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TABLE 17 

REGRESSION RESULTS FOR CU 



Data 


Model 


r 2 


B 0 




B l 


% 


n 


Elmendorf 


1-Lin 


.78 


-.75 


.96 


.559 


.028 


117 


Langley 


1-Lin 


.82 


-2.02 


1.38 


.651 


.041 


57 


Myrtle Beach 


1-Lin 


.92 


-2.66 


.88 


.737 


.022 


51 


Pensacola 


1-Lin 


.90 


-2.66 


.88 


.737 


.022 


118 


All Loc 


1-Lin 


.84 


-2.19 


.56 


.667 


.016 


343 


LanMyrPen 


1-Lin 


.89 


-2.73 


.62 


.713 


.017 


230 


All Loc 


1-Quac 


.84 


* 


* 


* 


* 


343 


















Corr Means 


2- Lin 


.91 


-1.30 


1.42 


.652 


.035 


36 


Verif Means 


2-Lin 


.99 


-2.70 


.48 


.638 


.016 


12 


All Loc Means 


2- Lin 


.92 


-1.93 


1.08 


.658 


.028 


48 


LMP Means 


2-Lin 


.94 


-2.40 


1.19 


.699 


.031 


36 


PWMA 2 -Dash-3 2 


2-Lin 


.86 


* 


* 


* 


* 


48 


PWMA-Dash-3 2 


2-Lin 


.86 


♦ 


* 


* 


* 


48 


PWMA 2 -Dash-3 


2-Lin 


.84 


* 


* 


* 


* 


48 



"“-denotes not included due to little or no r 2 improvement. 
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TABLE 18 

REGRESSION RESULTS FOR MG 



Data 


Model 


r 2 


B 0 


<Td 

B 0 


B l 


% 


n 


Elmendorf 


1-Lin 


.81 


-3.39 


.66 


.526 


.023 


116 


Langley 


1-Lin 


.85 


-3.12 


.99 


.647 


.036 


57 


Myrtle Beach 


1-Lin 


.82 


-2.97 


1.15 


.586 


.040 


51 


Pensacola 


1-Lin 


.87 


.20 


.62 


.648 


.024 


115 


All Loc 


1-Lin 


.76 


-1.70 


.49 


.578 


.018 


339 


LanMyrPen 


1-Lin 


.35 


-6.35 


.80 


.317 


.028 


231 


All Loc 


1-Quac 




♦ 


♦ 


* 


* 


339 


















Corr Means 


2-Lin 


.84 


-1.98 


1.53 


.584 


.046 


34 


Verif Means 


2-Lin 


.80 


-2.65 


2.16 


.619 


.097 


12 


All Loc Means 


2-Lin 


.84 


-2.06 


1.19 


.587 


.038 


46 


LMP Means 


2-Lin 


.84 


-1.57 


1.41 


.602 


.045 


34 


PWMA 2 -Dash-3 2 


2-Lin 


.79 


* 


* 


* 


♦ 


46 


PWMA-Dash-3 2 


2-Lin 


.79 


♦ 


* 


* 


* 


46 


PWMA 2 -Dash-3 


2-Lin 


.77 


* 


* 


* 


it 


46 



^-denotes not included due to little or no r 2 improvement. 
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